Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Explicitly clear the kv cache each time we eval tokens to match n_past. #1808

Merged
merged 1 commit into from
Jan 3, 2024
Merged

Explicitly clear the kv cache each time we eval tokens to match n_past. #1808

merged 1 commit into from
Jan 3, 2024

Conversation

manyoso
Copy link
Collaborator

@manyoso manyoso commented Jan 3, 2024

Describe your changes

Issue ticket number and link

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • I have added thorough documentation for my code.
  • I have tagged PR with relevant project labels. I acknowledge that a PR without labels may be dismissed.
  • If this PR addresses a bug, I have provided both a screenshot/video of the original bug and the working solution.

Demo

Steps to Reproduce

Notes

@manyoso manyoso requested a review from cebtenzzre January 3, 2024 17:28
@cebtenzzre cebtenzzre merged commit 96cee4f into main Jan 3, 2024
3 of 5 checks passed
@cebtenzzre cebtenzzre deleted the explicit_kv_clear branch January 3, 2024 19:06
@ThiloteE ThiloteE mentioned this pull request Jan 8, 2024
Closed
dpsalvatierra pushed a commit to dpsalvatierra/gpt4all that referenced this pull request Feb 16, 2024
@manyoso @dpsalvatierra
Explicitly clear the kv cache each time we eval tokens to match n_pas…
a0ee0af 
…t. (nomic-ai#1808)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cebtenzzre cebtenzzre cebtenzzre approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@manyoso @cebtenzzre